Parallel optimal pretesting of mixed-format questions for achievement tests

Abstract: When large achievement tests like national tests in school or admission tests for university are conducted regularly, the test-questions need to be pretested before being used in the test. Methods for assigning pretesting questions to examinees in an optimal way based on their ability have been developed. Most of these methods are intended for a situation where examinees arrive sequentially for being assigned to pretesting questions. However, several pretests (e.g., for national tests in Swedish schools or for högskoleprovet) are conducted in a way where all or many examinees conduct the test in parallel. In this talk, we develop an optimal design for such parallel pretest setups which can be implemented in real scenarios. In many real test situations, questions are of mixed format and our optimal design method can handle that. We discuss first the optimal designs for the 2-parameter logistic, the 3-parameter logistic, and the generalized partial credit model. Then, we consider the case of mixed-format tests where all these models are used to fit the data. The method we propose can also take different expected solve times into consideration. We investigate the efficiency gain of the method. Our investigations show that the proposed method is able to increase the efficiency of pretests considerably. The described method has been used for the Swedish national tests in mathematics.

This is a joint work with Ellinor Fackle-Fornius

machine learningprobabilitystatistics theory

Audience: researchers in the discipline

Gothenburg statistics seminar

Series comments: Gothenburg statistics seminar is open to the interested public, everybody is welcome. It usually takes place in MVL14 (http://maps.chalmers.se/#05137ad7-4d34-45e2-9d14-7f970517e2b60, see specific talk). Speakers are asked to prepare material for 35 minutes excluding questions from the audience.